TITLE OF THE INVENTION 


DEMULTIPLEXING A STATISTICALLY MULTIPLEXED MPEG TRANSPORT 
STREAM INTO CBR SINGLE PROGRAM TRANSPORT STREAMS 


BACKGROUND OF THE INVENTION 

The present invention relates to video compression, and more 
particularly to a simple method for demultiplexing a statistically 
multiplexed MPEG transport stream (MPTS) into constant bit rate (CBR) 

single program transport streams (SPTSs) without making any changes to 
the video elementary streams. 

The video elementd^ streams in the MPTS are all typically encoded 
with a variable bit rate (VBRX Only if the constant bit rate is too low does 
it becomes impossible to demultiplex a single stream from the MPTS 
without modifying the video elementary stream. Obviously this is the 
case if the average bit rate of the VBR stream is greater than the constant 
bit rate of the CBR stream. Even in this case, it is usually possible to 
ameliorate the potential visual artifacts by replacing B-type pictures with 
null B-type pictures. 

Prior efforts that achieve the demultiplexing of a statistically 
multiplexed MPTS into multiple CBR SPTSs do sckby modifying 
(transcoding) the video streams — while more general, it is a much more 
compute intensive process and can negatively impact ihe quality of the 
resultant video. Examples of such products are the CherryPicker 7000 
video re-multiplexer from Terayon Communication Systems located in 
Santa Clara, California, United States of America and possibly the PS5030 
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Multiplexer Module from PixStream Ih^orporated located in Waterloo, 


Qiitaiior^nadaT 


In a statistically multiplexed MPTS, video streams are encoded as 
VBR with a wide distribution of bit rates per picture. The transport 
packets (TPs) for a given video stream arrive in almost as bursty a pattern, 
though there is some smoothing. What is desired is to fit a complete SPTS 
with VBR video and CBR audio within a CBR network connection without 
recoding or modifying the video in any way. 


BRIEF SUMMARY OF THE INVENTION 

Accordingly the present invention provides a simple method for 
demultiplexing a statistically multiplexed MPEG transport stream into 
CBR single program transport streams without recoding or modifying the 
video in any way. The method starts sending a picture at a fixed interval, 
on the order of tens or hundreds of milliseconds, before it is to be decoded 
or as soon thereafter as possible. A separate logical smoothing buffer is 
used for each variable bit rate single program transport stream. When the 
decode time arrives the picture is transferred from the smoothing buffer for 
decoding. In the event there is buffer overflow, B-type frames are replaced 
with null B-type frames, which has the effect of a freeze frame by 
repeating the prior decoded picture, until the overflow condition ceases. 

The objects, advantages and other novel features of the present 
invention are apparent from the following detailed description when read 
in conjunction with the appended claims and attached drawing. 
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BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
Fig. 1 is an illustrative view of a statistically multiplexed MPEG 

Transport Stream having variable bit rate single program transport streams. 
Fig. 2 is an illustrative view of the relationship between the input 

variable bit rate single program transport stream and the constant bit rate 

single program transport stream produced using a smoothing buffer 

according to the present invention. 

Fig. 3 is an illustrative timing view showing the loading of the 

pictures into the smoothing buffer at the constant bit rate starting at a 

fixed time prior to the decode time, or as soon as possible thereafter, 

according to the present invention. 


DETAILED DESCRIPTION OF THE INVENTION 

^Referring now to Fig. 1 a statistically multiplexed MPEG Transport 
Stream (MPTS) having a plurality of programs of variable bit rates 
multiplexed together is illustrated. The shadowed packets represent a 
single program transport stream, Program 1, the packets of which all are a 
part of the same programNAs^shown between PCR 0 ms and PGR 1.8048 
ms Program 1 has one-third of the packets that are interleaved with those 
of other programs at an MPTS stream rate of 10 Mbs, i.e., 0.1504 ms per 
MTS packet. The bit rate for ProgramVover that interval is 3.33333 Mbs. 
Between the next two PCRs at 1.8048 ms N and 2.1056 ms are only two 
Program 1 packets with no interleaved packets sosthe Program 1 bit rate is 
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that of^the MTS stream, i.e., 10 Mbs. The different bit rates for Program 1 
depend upon, for example, image complexity and the type of MPEG frame 
— I, P or : 

The StTS(VBR) is separated from the MTS based upon the packet 
PIDs, and is input to a logical smoothing buffer at the desired constant bit 
rate to produce kn SPTS having a Constant Bit Rate (CBR). The SPTS 
(CBR) may then be decoded at the time indicated by the decode time 
stamp (DTS) for eacij frame to recover the original images in the video 
stream. 

As shown in Fig.Na for an SPTS (VBR) that has an Average Bit Rate 
(ABR) less than the desired CBR, there are times when the SPTS (VBR) 
exceeds the CBR. In the absence of the present invention this causes those 
pictures not to be decoded, resulting in repeated frames since the prior 
frame is repeated if the next frame is not decoded. The still frame remains 
until the next complete image onframe is available for the decoder at the 
time indicated by the associated decode time stamp (DTS). To prevent 
this undesired anomaly, the solution according to the present invention is 
to start loading the buffer early with tne packets for a frame so that when 
decode time comes the full data for the trame is available for decoding, as 
illustrated in Fig. 3 and described in more detail below. 

Buf n represents the smoothing buffer for variable bit rate (VBR) 
video elementary stream n within a statistically multiplexed MPEG 
transport stream (MPTS). The variable td n (j) is a\decode time for the j'th 


* • 

-5- 

picture of the video elementary stream n in a system target decoder. 
Decode time is measured in seconds and is with respect to a system clock 
PGR for the program containing elementary stream n. The variable tj[)) is 
a time at which the i'th transport stream (TS) packet of the j'th picture of 
elementary stream n enters the smoothing buffer Buf n . Again time is 
measured in seconds and is with respect to the system clock for the 
program containing elementary stream n. 1^ is based on the desired bit 
rate shown in Fig. 2 — it is the desired bit rate less the bit rate of the 
packets in the SPTS not carrying video. is a constant rate at which the 
TS packets with the video elementary stream n enter the smoothing buffer 
Buf n . The rate is given in units of bits per second. Finally 8 n is the earliest 
time before the decode time that a video elementary stream TS packet can 
enter the smoothing buffer Buf n . Again time is given in seconds. 

The buffer size is given by I Buf n I = R n 8 n . TS packets containing 
video elementary stream n enter Buf n as early as possible subject to the 
following constraints: 

1. The rate at which the packets can enter Buf n is limited by 1^. 

2. For any picture j of video elementary stream n and for all TS packets 
containing picture j, t n ! (j) + 5 n * tdjj). 

3. At time td n (j) all of the packets t^fj) are removed. 

4. If the buffer Buf n is full, i.e., contains R n 6 n bits, then no new packets 
can enter. 
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The buffer model is satisfied if the buffer Buf n never overflows or 
underflows. An overflow or underflow occurs when for any picture j of 
video elementary stream n and any TS packet containing picture j, 

Og) > td n o). \ 

In the event of overflow a B-type frame is replaced by a null B-type 
frame until the overflow ceases, v\mich has the effect of repeating the prior 
video frame by the decoder, as described in co-pending U.S. Patent 
Application No. 09/113,669 entitled ^Readjustment of Bit Rates When 
Switching Between Compressed Video\streams" by Douglas C. Stevens et 
*h 

The above is illustrated in Fig. 3 where a series of frames of variable 
bit rates is shown. At time 5 before the DTS for frame 1, frame 1 is loaded 
into the buffer. The buffer may be thought of as having a plurality of equal 
capacity slots between consecutive DTS times. Since frame 1 has fewer 
bits than the capacity of one slot, there is a gap before frame 2 is loaded 
into the buffer, again at time 5 before the DTS for frame 2. Likewise frame 
3 is loaded at time 8 before DTS 3 . Frame 3 has more bits than fit into one 
slot, so that frame 4 is loaded into the buffer as soon as possible thereafter. 
Then frame 5 is loaded into the buffer as soon as possible. The end of 
frame 5 almost exceeds the DTS time for frame 5 and the buffer is in 
danger of overflowing. However the next few frames 6, 7 and 8 each have 
fewer bits than one slot so that the capacity of the buffer is alleviated. 

Thus the present invention provides for demultiplexing a 
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statistically multiplexed MPEG transport stream into a constant bit rate 
single program transport stream using a smoothing buffer with the video 
pictures in the stream being loaded into the smoothing buffer at the 
desired constant bit rate as early as possible up to a fixed time interval 
before the pictures need to be decoded. 


